Vehicle sensor alignment using photogrammetry

ABSTRACT

A method is provided for mounting a plurality of radar units to a vehicle. The method involves determining, for each radar unit, a measured pitch direction and a measured yaw direction based on data obtained using a photogrammetry system. The method also involves determining yaw angles between at least two of the radar units based on at least one of the measured yaw directions. The method also involves determining, for each radar unit, a pitch offset and a yaw offset. The method also involves adjusting at least one of the radar units based on at least one of the determined pitch offsets and at least one of the determined yaw offsets. Also provided is a device for performing the method.

BACKGROUND

Unless otherwise indicated herein, the materials described in thissection are not prior art to the claims in this application and are notadmitted to be prior art by inclusion in this section.

Radio detection and ranging (RADAR) systems can be used to activelyestimate distances to environmental features by emitting radio signalsand detecting returning reflected signals. Distances to radio-reflectivefeatures can be determined according to the time delay betweentransmission and reception. The radar system can emit a signal thatvaries in frequency over time, such as a signal with a time-varyingfrequency ramp, and then relate the difference in frequency between theemitted signal and the reflected signal to a range estimate. Somesystems may also estimate relative motion of reflective objects based onDoppler frequency shifts in the received reflected signals.

Directional antennas can be used for the transmission and/or receptionof signals to associate each range estimate with a bearing. Moregenerally, directional antennas can also be used to focus radiatedenergy on a given field of view of interest. Combining the measureddistances and the directional information allows for the surroundingenvironment features to be mapped. The radar sensor can thus be used,for instance, by an autonomous vehicle control system to avoid obstaclesindicated by the sensor information.

Some example vehicle radar systems may be configured to operate at anelectromagnetic wave frequency of 77 Giga-Hertz (GHz), which correspondsto a millimeter (mm) wave electromagnetic wave length (e.g., 3.9 mm for77 GHz). These radar systems may use antennas that can focus theradiated energy into tight beams in order to enable the radar system tomeasure an environment with high accuracy, such as an environment aroundan autonomous vehicle. Such antennas may be compact (typically withrectangular form factors), efficient (i.e., with little of the 77 GHzenergy lost to heat in the antenna or reflected back into thetransmitter electronics), and low cost and easy to manufacture (i.e.,radar systems with these antennas can be made in high volume).

SUMMARY

Disclosed herein are embodiments that relate to vehicle sensor alignmentusing photogrammetry. In one example, a method is provided that involvesmounting a plurality of radar units to a vehicle. The method alsoinvolves determining, for each radar unit, a measured pitch directionand a measured yaw direction based on data obtained using aphotogrammetry system. The method also involves determining yaw anglesbetween at least two of the radar units. A particular yaw angle betweentwo particular radar units may be based on a difference between themeasured yaw directions of the two particular radar units. The methodalso involves determining, for each radar unit, a pitch offset based ona difference between a target pitch direction and the measured pitchdirection of the radar unit. The method also involves determining, foreach radar unit, a yaw offset based at least in part on a differencebetween a target yaw angle and one of the determined yaw angles. Themethod also involves adjusting orientation of at least one of the radarunits based on at least one of the determined pitch offsets and at leastone of the determined yaw offsets.

In another example, a device is provided that includes data storagestoring data generated by a photogrammetry system. The data may indicatemeasured directions of a plurality of radar units mounted to a vehicle.The device also includes a computing unit configured to performoperations comprising using the data in the data storage to determine,for each radar unit, a measured pitch direction and a measured yawdirection. The operations also comprise determining yaw angles betweenat least two of the radar units. A particular yaw angle between twoparticular radar units may be based on a difference between the measuredyaw directions of the two particular radars. The operations alsocomprise determining, for each radar unit, a yaw offset based at leastin part on a difference between a target yaw angle and one of thedetermined yaw angles. The operations also comprise determining, foreach radar unit, a pitch offset based on a difference between a targetpitch direction and the measured pitch direction of the radar unit. Theoperations also comprise providing mounting instructions for mountingthe radar units. The mounting instructions are based on the determinedpitch offsets and the determined yaw offsets.

In yet another example, a computing device is provided. The computingdevice may include a processor and a computer readable medium havingstored thereon program instructions that when executed by the processorcause the computing device to perform functions. The functions includeobtaining data generated by a photogrammetry system. The data maypertain to a plurality of radar units mounted to a vehicle. Thefunctions also comprise determining, based on the obtained data,measured yaw angles between at least two of the radar units. Aparticular measured yaw angle between two particular radar units may bebased on a difference between yaw directions measured for the twoparticular radar units. The functions also comprise determining, foreach radar unit, a yaw offset based at least in part on a differencebetween a target yaw angle and one of the measured yaw angles. Thefunctions also comprise determining, for each radar unit, a measuredpitch direction based on the obtained data. The functions also comprisedetermining, for each radar unit, a pitch offset based on a differencebetween a target pitch direction and the measured pitch direction.

In still another example, a system is provided that includes means formounting a plurality of radar units to a vehicle. The system alsoincludes means for determining, for each radar unit, a measured pitchdirection and a measured yaw direction based on data obtained using aphotogrammetry system. The system also includes means for determiningyaw angles between at least two of the radar units. A particular yawangle between two particular radar units may be based on a differencebetween the measured yaw directions of the two particular radar units.The system also includes means for determining, for each radar unit, apitch offset based on a difference between a target pitch direction andthe measured pitch direction of the radar unit. The system also includesmeans for determining, for each radar unit, a yaw offset based at leastin part on a difference between a target yaw angle and one of thedetermined yaw angles. The system also includes means for adjustingorientation of at least one of the radar units based on at least one ofthe determined pitch offsets and at least one of the determined yawoffsets.

The foregoing summary is illustrative only and is not intended to be inany way limiting. In addition to the illustrative aspects, embodiments,and features described above, further aspects, embodiments, and featureswill become apparent by reference to the figures and the followingdetailed description.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 illustrates a layout of radar sectors, according to an exampleembodiment.

FIG. 2 illustrates beam steering for a sector for a radar unit,according to an example embodiment.

FIG. 3 illustrates a block diagram for mounting a radar unit, accordingto an example embodiment.

FIG. 4 illustrates a computing device, according to an exampleembodiment.

FIG. 5 illustrates a method, according to an example embodiment.

DETAILED DESCRIPTION

In the following detailed description, reference is made to theaccompanying figures, which form a part hereof. In the figures, similarsymbols typically identify similar components, unless context dictatesotherwise. The illustrative embodiments described in the detaileddescription, figures, and claims are not meant to be limiting. Otherembodiments may be utilized, and other changes may be made, withoutdeparting from the scope of the subject matter presented herein. It willbe readily understood that the aspects of the present disclosure, asgenerally described herein, and illustrated in the figures, can bearranged, substituted, combined, separated, and designed in a widevariety of different configurations, all of which are explicitlycontemplated herein.

The following detailed description relates to systems, devices, andmethods for vehicle sensor alignment using photogrammetry. Althoughexample embodiments herein may involve radar sensors, the methods andsystems herein can be used with any vehicle-mounted sensors, such asLIDARs, cameras, motion sensors, or microphones, among otherpossibilities. Vehicular radar systems may feature multiple radar unitswith each radar unit having an associated field of view. Conventionalapproaches for alignment of the various radar units can betime-consuming, labor-intensive procedures involving precisemeasurements and expensive tools. The devices and methods presentedherein may allow precise alignment of radar units in a more timeefficient and equipment-efficient manner.

Traditionally, vehicular radar systems have featured a single radar unitlooking in a forward direction. These traditional radar systems wouldalso typically only direct the radar beam in a single direction. When avehicle has only a single radar unit with a beam that only was directedin a single direction, precise placement of the radar unit may be lessof a concern than for systems using multiple radar units with steerablebeams. For example, a traditional vehicular radar system may only beconfigured to detect the presence of another vehicle directly in frontof the vehicle having the radar system. Thus, as long as a radar beamgenerally points in the forward direction, the traditional radar systemmay be able to detect the vehicle.

More advanced radar systems may be used with a vehicle in order toobtain a wider field of view than just directly in front of the vehicle.For example, it may be desirable either for a radar to be able to steera radar beam or for a vehicle to feature multiple radar units pointingin different directions. Additionally, the radar units may be configuredto detect more than just other vehicles located directly in front of thevehicle with the radar unit. Thus, the radar system may be able tointerrogate different regions than just the region in front of the car.In some examples, multiple radar units may be combined with steerableradar beams to further increase the interrogation region and the imagingresolution of the vehicular radar system.

An example radar system for use with the present disclosure may includea multi-sector 90-degree field of view radar antenna architecture thatmay enable an antenna to both scan across approximately 90 degrees ofthe azimuth plane (e.g., the horizontal plane, the yaw plane, etc.)while also being mountable on various surfaces of a vehicle. Having aradar antenna with a 90 degree field of view may enable a radar systemto scan a full 360 azimuth (yaw) plane by having four radar units eachconfigured to scan one 90 degree non-overlapping sector. Therefore, theexample disclosed radar system may be able to steer a radar beam tointerrogate the entire region in the azimuth (yaw) plane of the vehicle.For example, four such radars located on four corners of a car mayprovide a full 360 degree coverage around the car. Such a system may aidin autonomous driving of a vehicle.

When each radar unit can scan or span a 90 degree region, placing 4radar units on a vehicle may enable the vehicle to scan a beam over thefull 360 azimuth (yaw) plane. Each of the four radar units may beconfigured to scan a beam over one sector (i.e. one quarter of theazimuth/yaw plane) and thus the entire plane may be scanned by thecombination of the four radar units. In various examples, the placementof the radar units may be adjusted depending on the specific vehicle,the requirements of the radar system, or other design criteria. In someadditional examples, the radar units may be configured to scan a regionof an angular width that is not 90 degrees. For example, some radarunits may scan 30 degrees, 120 degrees, or another angle. Further, insome examples, the radar units on the vehicle may scan less than thefull 360 azimuth (yaw) plane.

In some examples, the radar sectors may be defined based on where theradar units may be mounted on the vehicle. In one example, one radarunit may be mounted in each of the side mirrors of the vehicle. Theother two radar units may be mounted adjacent to or behind thetaillights of the vehicle. In this example, the quadrants may be definedbased on axes where one axis aligns with the direction of vehicularmotion (e.g., front to back) and the other axis aligns with the middleof the vehicle from left to right. In another example, the radar unitsmay be mounted in order to have one pointing forward, one pointingbackward, and one pointing to each side. In this second example, theaxes of the quadrants may be at a 45 degree angle to the direction ofmotion of the vehicle. Additionally, the radar unit may be mounted ontop of the vehicle.

A multi-sector 90-degree field of view radar antenna architecture couldalso be able to steer the radar beams emitted from each radar unit. Theradar beams may be steered by the radar units in various ways. Forexample, in some embodiments, the radar units may be able to steer thebeam in an approximately continuous manner across the 90 degree field ofview for the respective antenna or the radar units may be configuredwith sectoral sub beams spanning the 90 degrees. In other embodiments,the radar units may be able to steer the radar beam to predetermineddirections within the 90 degree field of view for the respectiveantenna.

FIG. 1 illustrates an example layout of radar sectors for a vehicle 102.As shown in FIG. 1, each of the radar sectors may have an angular widthapproximately equal to the scanning range of the radar units (as will bedescribed with respect to FIG. 2). For example, the sectors of FIG. 1divide the azimuth (yaw) plane around the vehicle 102 into 90 degreesectors. However, in examples where the radar units are configured toscan a radar beam over a different angle than 90 degrees, the width andnumber of sectors may change.

As shown in FIG. 1, the radar sectors may align with the axes (112 a and112 b) of the vehicle 102. For example, there may be a front left, frontright, rear left, and rear right sector defined by the midpoints of thevehicle 102. Because each sector corresponds to one radar unit, eachradar unit may be configured to scan across one sector. Further, becauseeach example radar unit of FIG. 1 has a scanning angle of approximately90 degrees, each radar unit scans a region that approximately does notoverlap with the scanning angle of any other radar unit. The layout ofradar sectors shown in FIG. 1 is one example. Other possible layouts ofradar sectors are possible as well.

In order to achieve radar sectors defined by the midpoints of thevehicle 102, each radar unit may be mounted at a 45 degree angle withrespect to the two axes of the vehicle 102, such that the radar unit hasa yaw pointing direction aligned approximately between the two axes (112a and 112 b). By mounting the radar units at a 45 degree angle (e.g.,the yaw direction) with respect to the two axes of the vehicle 102, a 90degree scan of the radar unit would scan from one vehicle axis to theother vehicle axis. For example, a radar unit mounted at a 45 degreeangle to the axes in side mirror unit 104 (e.g., having yaw direction124) may be able to scan the front left sector (i.e. from the verticalaxis 112 a through the front of the vehicle 102 to the axis 112 b thatruns through the side of the vehicle). An additional radar unit may bemounted at a 45 degree angle to the axes in side mirror unit 106 (e.g.,having yaw direction 126) may be able to scan the front right sector. Inorder to scan the back right sector, for example, a radar unit may bemounted at or near taillight unit 110 (e.g., having yaw direction 130).Additionally, in order to scan the back left sector, a radar unit may bemounted at or near taillight unit 108 (e.g., having yaw direction 128),for example. The radar unit placements shown in FIG. 1 are merely oneexample. In various other examples, the radar units may be placed inother locations, such as on top of the vehicle, or within or behindother vehicle components. Further, the sectors may also be defineddifferently in various embodiments. In one example, the sectors may beat a 45 degree angle with respect to the vehicle. In this example, oneradar unit may face forward, another backward, and the other two to thesides of the vehicle.

In some examples, all the radar units of vehicle 102 may be configuredwith the same scanning angle. The azimuth (yaw) plane around the vehicleis equal to a full 360 degrees. Thus, for example, if each radar unit isconfigured with the same scanning angle, then the scanning angle for theradar units would be equal to approximately 360 divided by the number ofradar units on the vehicle. Thus, for full azimuth (yaw) plane scanning,a vehicle 102 with one radar unit would need that radar unit to be ableto scan over the full 360 degrees.

If the vehicle 102 had two radar units, each could scan approximately180 degrees, for example. With three radar units, each could beconfigured to scan 120 degrees, for example. With four radar units, asshown in FIG. 1, each may scan approximated 90 degrees. Five radar unitsmay be configured on the vehicle 102 and each may be able to scan 72degrees. Further, six radar units may be configured on the vehicle 102and each may be able to scan approximately 60 degrees.

The number of radar units may be chosen based on a number of criteria,such as ease of manufacture of the radar units, vehicle placement, orother criteria. For example, some radar units may be configured with aplanar structure. The planar radar unit may be mountable at variouspositions on a vehicle. For example, a vehicle may have a dedicatedradar housing mounted on the top of the vehicle. The radar housing maycontain various radar units. However, in other embodiments, radar unitsmay be placed within the vehicle structure.

When radar units are located within the vehicle structure, each may notbe visible from outside the vehicle without removing parts of thevehicle. Thus, adding radar units to the vehicle may not alter thevehicle aesthetically, cosmetically, or aerodynamically. For example,radar units may be placed under vehicle trim work, under bumpers, undergrills, within housings for lights, within side mirrors, or otherlocations as well. In some embodiments, it may be desirable to placeradar units in positions where the object covering the radar unit is atleast partially transparent to radar wavelengths. For example, variousplastics, polymers, and other materials may form part of the vehiclestructure and cover the radar units, while allowing the radar signal topass through.

Additionally, in some embodiments, the radar units may be configuredwith different scanning ranges for different radar units. For example,in some embodiments a specific radar unit with a wide scanning angle maynot be able to be placed on the vehicle in the proper location. Thus, asmaller radar unit, with a smaller scanning angle may be placed in thatlocation. However, other radar units may be able to have larger scanningangles. Therefore, the total scanning angle of the radar units may addup to 360 degrees (or more) and provide full 360 degree azimuthal (yaw)scanning. For example, a vehicle may have 3 radar units that each scanover 100 degrees and a fourth radar unit that scans over 60 degrees.Thus, the radar units may be able to scan the full azimuth (yaw) plane,but the scanning sectors may not be equal in angular size.

FIG. 2 illustrates example beam steering for a sector for a radar unit200. The radar unit 200 may be configured with a steerable beam, i.e.,the radar unit 200 may be able to control a direction in which the beamis radiated. By controlling the direction in which the beam is radiated,the radar unit 200 may be able to direct radiation in a specificdirection in order to determine radar reflections (and thus objects) inthat direction. In some embodiments, the radar unit 200 may be able toscan a radar beam in a continuous manner across the various angles ofthe azimuth (yaw) plane. In other embodiments, the radar unit 200 may beable to scan the radar beam in discrete steps across the various anglesof the azimuth (yaw) plane.

The example radar unit 200 in FIG. 2 has a radar beam 206 that can besteered across a plurality of different angles. As shown in FIG. 2, theradar beam 206 may have a half-power beamwidth of approximately 22.5degrees. The half-power beamwidth describes the width, measured indegrees, of a main lobe of the radar beam 206 between two points thatcorrespond to half the amplitude of the maximum of the radar beam 206.In various embodiments, the half-power beamwidth of the radar beam 206may be different than 22.5 degrees. Additionally, in some embodiments,the half-power beamwidth of the radar beam 206 may change depending onthe angle at which the radar beam 206 is pointed. For example, thehalf-power beamwidth of the radar beam 206 may be narrower when theradar beam 206 is pointed more closely to the orthogonal 204 a (i.e.broadside) direction to the radiating surface and widen and the radarbeam 206 is steered away from the orthogonal direction 204 a.

In some examples, the orthogonal 204 a direction can be described as thepointing or reference direction of the radar unit 202. For example, ifthe yaw or azimuthal plane of a vehicle using radar unit 202 issubstantially parallel to the page of FIG. 2, then the component of theorthogonal direction 204 a substantially parallel to the page (as shown)could be described as the yaw direction of the radar unit 202. Further,in this example, the component of the orthogonal direction 204 a along aplane substantially perpendicular to the page and also substantiallyparallel to the orthogonal direction 204 a could be described as thepitch direction or the elevational direction of the radar unit 202.Thus, in this example, such reference directions (e.g., yaw directionand pitch direction) may indicate the orientation of the radar unit 202relative to the vehicle on which the radar unit 202 is mounted.

Continuing with the example shown in FIG. 2, the radar beam may be ableto be steered to four different angles. The steering angle may bemeasured with respect to the orthogonal 204 a (i.e. broadside) directionto the radiating surface. The beam may be steered to +36 degrees at 204c and −36 degrees at 204 e. Also, the beam may be steered to +12 degreesat 204 b and −12 degrees at 204 d. The four different angles mayrepresent the discrete angles to which the radar beam 206 may besteered. In some additional examples, the radar beam may be able to besteered to two angles simultaneously. For example, the radar beam may besteered to both +12 and −12 degrees at the same time. This may result ina beam that is overall steered in the direction of the sum of the angles(e.g. −12+12=0, thus the beam in this example would be in the broadsidedirection 204 a). However, when a radar beam is steered at twodirections at once, the half-power beamwidth of the radar beam may bewidened. Thus, a radar resolution may decrease.

By steering the radar beam 206 to each of angles 204 b-204 e, the full90 degree field of view can be scanned. For example, when the radar beam206 is steered to +36 degrees 204 c, the half-power beamwidth of theradar beam 206 will cover from +47.25 degrees to +24.75 degrees (asmeasured from the broadside direction 204 a). Additionally, when theradar beam 206 is steered to −36 degrees 604 e, the half-power beamwidthof the radar beam 206 will cover from −47.25 degrees to −24.75 degrees.Further, when the radar beam 206 is steered to +12 degrees 204 b, thehalf-power beamwidth of the radar beam 206 will cover from +23.25degrees to +0.75 degrees. And finally, when the radar beam 206 issteered to −12 degrees 204 d, the half-power beamwidth of the radar beam206 will cover from −23.25 degrees to −0.75 degrees. Thus, the radarbeam 206 will effectively be able to scan (e.g., by selectively enablingor disabling the four beams spanning the angular width) from −47.25 to+47.25 degrees, covering a range of 95 degrees. It is noted that thenumber of steering angles, the direction of the steering angles, and thehalf-power beamwidth of the radar beam 206 may be varied depending onthe specific example.

For example, and further discussed below, a radar beam of a radar unitmay be configured to only scan a 60 degree region. If a radar unit canscan a 60 degree region, six radar units may be used to scan a full 360azimuth (yaw) plane. However, if the radar unit can scan 90 degrees,four radar units may scan the full 360 azimuth (yaw) plane.

FIG. 3 illustrates mounting a radar unit 312 to a mounting structure302, in accordance with at least some embodiments herein. The mountingstructure 302 is one example way the various radar units could bemounted to a vehicle. However in some embodiments, the radar unit 312and the mounting structure 302 may be integrated as one unit and notseparate components as shown in FIG. 3. Further, in some embodiments,the various components shown in FIG. 3 may have different dimensions orshapes than those shown.

A mounting structure may be located at each location where a radar unitmay be coupled to a vehicle. For example, each radar unit mountinglocation (such as locations 104, 106, 108, and 110 of FIG. 1) may have amounting structure, similar to the mounting structure 302, to which aradar unit may be mounted. As shown in FIG. 3, the mounting structure302 may include mounting holes 304, 306, 308, and 310 configured to bothalign the radar unit 312 when mounted as well as allow the radar unit tobe coupled to the mounting structure 302 with attachment devices such asadhesives, bolts, washers, spherical washers, spherical nuts, shims,and/or screws, among other possibilities.

For example, as shown, the radar unit 312 may include mounting holes314,316, 318, and 320. Thus, as shown, bolts 324, 326, 328 could bepositioned (or fastened), respectively, through holes 314, 316, 318 ofthe radar unit along dotted lines 354, 356, 358 and into (or through)holes 304, 306, and 308 of the mounting structure. Through this process,for example, the radar unit 312 could be mounted onto the mountingstructure 302 that is included in the vehicle in line with thediscussion above.

Alternatively or additionally, in some examples, more or fewercomponents than those shown could be used to couple the radar unit 312with the mounting structure 302. In one example, another bolt (notshown) could be positioned or fastened through holes 320 and 310similarly to bolts 324, 326, and 328. In another example, one or morespherical washers (not shown) and/or spherical nuts (not shown) could bepositioned between the radar unit 312 and one or more of shims 344, 346,348. A spherical washer may be a washer having a curved surfaceconfigured to align with a corresponding curved surface of a sphericalnut. For instance, a spherical nut (not shown) could be positioned alongdotted line 334 between radar unit 312 and shim 344, and a sphericalwasher (not shown) could be positioned between the spherical nut andshim 344. With such arrangement, for example, an angle between dottedline 334 (e.g., path of bolt) and the spherical nut (not shown) couldvary within a threshold range of angles, as the curved surface of thespherical washer (not shown) can slide along the curved surface of thespherical nut (not shown), thereby reducing the likelihood of bindingbetween bolt 324 and shim 344. Other example arrangements that involvemore or fewer fastening components than those shown are possible as well(e.g., spherical washer positioned between shim 344 and mountingstructure 302, etc.).

In the example shown in FIG. 3, for each bolt, one or more shims couldbe positioned along the path of the bolt between radar unit 312 andmounting structure 302. A shim (e.g., washer, wedge, etc.) could be madefrom any solid material and can be used to align components, fitcomponents, or mitigate damage caused by friction between components(e.g., friction between radar unit 312 and mounting structure 302). Forexample, FIG. 3 shows shims 344 and 354 positioned between the radarunit 312 and the mounting structure 302 along dotted line 334 of thebolt 324. Thus, bolt 324 could be fastened through shims 344, 354 andinto hole 304. Similarly, a shim 346 is positioned along dotted line 336of bolt 326, and shims 348, 358 are positioned along dotted line 338 ofbolt 328. Other arrangements and configurations are possible as well formounting radar unit 312 to mounting structure 302.

As noted above, when mounting structure 302 is disposed on a vehiclesuch as the vehicle 102 of FIG. 1, the orientation of mounting structure302 could vary due to manufacturing or assembly differences betweendifferent vehicles. For example, the orientation of mounting structure302 may vary in terms of pitch (elevational) direction 362 a, rolldirection 362 b, and/or yaw (azimuthal) direction 362 c along therespective arrows shown in FIG. 3. In some embodiments, it may bedesirable for a radar unit to be mounted within a threshold range arounda desired direction. As an example, each of pitch direction 372 a, rolldirection 372 b, and/or yaw direction 372 c of radar unit 312 may beassociated with a threshold range of ±1 degree from the desired pitchdirection, roll direction, and/or yaw direction. Therefore, it may bedesirable to adjust or offset pitch direction 372 a, roll direction 372b, and/or yaw direction 372 c of radar unit 312 relative to,respectively, pitch direction 362 a, roll direction 362 b, and/or yawdirection 362 c of mounting structure 302.

Within examples herein, these adjustments or offsets could be achievedby varying the quantity of shims used with one or more of the bolts. Forexample, as shown, bolt 326 is coupled to only one shim 346, whereasbolts 324 and 328 are coupled to two bolts each (e.g., shims 344 and 354for bolt 324, and shims 348 and 358 for bolt 328). In this example, theradar unit 312 would have a yaw offset relative to the mountingstructure 302.

Thus, in some embodiments, the present disclosure may provide forvarying the quantities of shims used for each bolt used to couple aradar unit to a mounting structure. By doing so, for example, a desiredyaw offset or pitch offset between the radar unit and the mountingstructure could be achieved, and the radar unit could be pointed towardsa desired direction in line with the discussion above.

In one example, to adjust the yaw direction 372 c of radar unit 312,more or fewer shims could be used for bolt 326 than the one shim 346shown in FIG. 3. Alternatively or additionally, for instance, more orfewer shims could be used for both bolts 324, 328 than those shown inFIG. 3.

In another example, to adjust the pitch direction 372 a, more or fewershims could be used for bolt 328 than the two shims 348, 358 shown inFIG. 3. Alternatively or additionally, for instance, more or fewer shimscould be used for both bolts 324, 326 than those shown in FIG. 3.

In some embodiments, a computing device may implement the disclosedmethods as computer program instructions encoded on a non-transitorycomputer-readable storage media in a machine-readable format, or onother non-transitory media or articles of manufacture. The computingdevice may be integrated with a photogrammetry system or it may be aseparate computing device in communication with the photogrammetrysystem.

FIG. 4 illustrates a functional block diagram of a computing device 400in communication with a photogrammetry system 480, according to anexample embodiment. The computing device 400 can be used to perform someor all of the functions discussed above in connection with FIG. 5.

The computing device 400 can be or include various types of devices,such as, for example, a server, personal computer, mobile device,cellular phone, or tablet computer. In a basic configuration 402 (e.g.,computational unit, etc.), the computing device 400 can include one ormore processors 410 and system memory 420. A memory bus 430 can be usedfor communicating between the processor 410 and the system memory 420.Depending on the desired configuration, the processor 410 can be of anytype, including a microprocessor (μP), a microcontroller (μC), or adigital signal processor (DSP), among others. A memory controller 415can also be used with the processor 410, or in some implementations, thememory controller 415 can be an internal part of the processor 410.

Depending on the desired configuration, the system memory 420 can be ofany type, including volatile memory (such as RAM) and non-volatilememory (such as ROM, flash memory). The system memory 420 may includeprogram instructions that, when executed by the processor 410, may causethe device 400 to perform functions described herein (e.g., functions ofthe method 500 in FIG. 5). For example, the functions may include usingdata stored in data storage 440 to determine a measured pitch directionand a measured yaw direction for each radar unit coupled to a vehicle,in line with the discussion above. Thus, for example, the data stored indata storage 440 may include data generated by photogrammetry system480, such as the measured pitch and yaw directions, or measured yawangles between radar units mounted to a vehicle (e.g., radar units 202and 312 of FIGS. 2 and 3).

The system memory 420 can include one or more applications 422 andprogram data 424. The application(s) 422 can include an index algorithm423 that is arranged to provide inputs to the electronic circuits. Theprogram data 424 can include content information 425 that can bedirected to any number of types of data. The application 422 can bearranged to operate with the program data 424 on an operating system.The program data 424, for example, may include the data generated by thephotogrammetry system 480, in line with the discussion above. In oneexample, the program instructions, when executed by processor 410, couldcause the device 400 to receive the data generated by the photogrammetrysystem 480 via communication interface 470, and could cause the device400 to store the received data in data storage 440.

The computing device 400 can have additional features or functionality,and additional interfaces to facilitate communication between the basicconfiguration 402 and any devices and interfaces. For example, datastorage devices 440 can be provided including removable storage devices442, non-removable storage devices 444, or both. Examples of removablestorage and non-removable storage devices include magnetic disk devicessuch as flexible disk drives and hard-disk drives (HDD), optical diskdrives such as compact disk (CD) drives or digital versatile disk (DVD)drives, solid state drives (SSD), and tape drives. Computer storagemedia can include volatile and nonvolatile, non-transitory, removableand non-removable media implemented in any method or technology forstorage of information, such as computer readable instructions, datastructures, program modules, or other data.

The system memory 420 and the storage devices 440 are examples ofcomputer storage media. Computer storage media includes, but is notlimited to, RAM, ROM, EEPROM, flash memory or other memory technology,CD-ROM, DVDs or other optical storage, magnetic cassettes, magnetictape, magnetic disk storage or other magnetic storage devices, or anyother non-transitory computer readable medium that can be used to storethe desired information and that can be accessed by the computing device400.

The computing device 400 can also include output interfaces 450 that caninclude a graphics processing unit 452, which can be configured tocommunicate with various external devices, such as display devices 490or speakers by way of one or more A/V ports or a communication interface470. The communication interface 470 can include a network controller472, which can be arranged to facilitate communication with one or moreother computing devices, such as photogrammetry system 480, over anetwork communication by way of one or more communication ports 474. Thecommunication connection is one example of a communication media.Communication media can be embodied by computer-readable instructions,data structures, program modules, or other data in a modulated datasignal, such as a carrier wave or other transport mechanism, andincludes any information delivery media. A modulated data signal can bea signal that has one or more of its characteristics set or changed insuch a manner as to encode information in the signal. By way of example,and not limitation, communication media can include wired media such asa wired network or direct-wired connection, and wireless media such asacoustic, radio frequency (RF), infrared (IR), and other wireless media.

The computing device 400 can be implemented as a portion of a small-formfactor portable (or mobile) electronic device such as a cell phone, apersonal data assistant (PDA), a personal media player device, awireless web-watch device, a personal headset device, an applicationspecific device, or a hybrid device that include any of the abovefunctions. The computing device 400 can also be implemented as apersonal computer including both laptop computer and non-laptop computerconfigurations.

The disclosed methods can be implemented as computer programinstructions encoded on a non-transitory computer-readable storagemedium in a machine-readable format, or on other non-transitory media orarticles of manufacture. The computer program product includes acomputer program for executing a computer process on a computing device,arranged according to some disclosed implementations.

The one or more programming instructions can be, for example, computerexecutable instructions. A computing device (such as the computingdevice 400 of FIG. 4) can be configured to provide various operations inresponse to the programming instructions conveyed to the computingdevice by one or more of the computer-readable medium, the computerrecordable medium, and the communications medium.

Photogrammetry system 480 may include one or more devices as well as oneor more software components configured to provide data to the computingdevice 400 via the communication interface 470. The data may indicatemeasured pitch directions, roll directions, and/or yaw directions of aplurality of radar units (or mounting structures) coupled to a vehiclesuch as the vehicle 102 of FIG. 1. For example, the photogrammetrysystem 480 may include one or more imaging devices (e.g., cameras, depthcameras, 3D cameras, etc.) and/or one or more remote sensing devices(e.g., Infrared sensors, ultrasound sensors, etc.), and may use thesedevices to detect, measure, or record 3D orientations of varioussurfaces or objects on or around a vehicle such as vehicle 102.

In one embodiment, photogrammetry system 480 may detect tags (e.g.,stickers, markers, etc.) positioned on surfaces of the various radarunits (e.g., radar unit 302), vehicle (e.g., vehicle 102), and/or groundaround the vehicle, among other possibilities. In this embodiment,photogrammetry system 480 may capture images of the tags from variousangles, and may then use the captured images as a basis to determine thevarious directions (e.g., yaw direction, pitch direction, etc.) of thetagged surfaces (e.g., surface of radar unit 312 or mounting structure302, etc.) relative to the vehicle (e.g., vehicle 102). This data couldbe used according to example methods and systems herein to determineorientation adjustments (e.g., pitch offsets, yaw offsets, etc.) formounting radar units to a vehicle in line with the discussion above.Other configurations for photogrammetry system 480 are possible as well(e.g., tag-less photogrammetry, stereophotogrammetry, etc.). Further,other operations of photogrammetry system 480 are possible as well andare described in greater detail with exemplary embodiments herein.

In some embodiments, computing device 400 could be configured to requestthe data from the photogrammetry system 480 via the communicationinterface 470, and responsively receive the data. Further, in someembodiments, computing device 400 could be configured to store thereceived data in storage devices 440 for later use by processor 410 inaccordance with the methods of the present disclosure.

FIG. 5 is an example method for vehicle sensor alignment usingphotogrammetry. Moreover, the method 500 of FIG. 5 will be described inconjunction with FIGS. 1-4. A vehicular radar system may be configuredto interrogate the region around the vehicle via multiple radar units.To interrogate the region around the vehicle, the radar system maytransmit the radar beam in a given direction. The transmitted beam mayreflect off objects in the region, and the radar unit may receive thesereflections. The received reflections may allow the radar system and acomputer to determine what objects are located near the vehicle. Notonly may objects themselves be determined, but the location (i.e. angleand range to objects) may be determined as well. In order to operatecorrectly, radar units of the radar system need to both be placed incorrect locations and have fairly precise alignment. Accordingly, method500 could be used for mounting a plurality of radar units to a vehiclein line with the discussion above for FIGS. 1-4.

At block 502, the method 500 includes mounting a plurality of radarunits to a vehicle. Each of the radar units could be mounted at amounting location, such as locations 104, 106, 108, 110 of FIG. 1, andcould be configured to scan a surrounding environment of the vehicle,similarly to radar units 202 and 312 of FIGS. 2 and 3. In oneembodiment, the mounting at block 502 may be performed by a robotic armor other electromechanical device that is operated by a computing devicesuch as device 400 of FIG. 4. In another embodiment, the mounting atblock 502 may involve providing mounting instructions to a humanoperator for mounting the radar units. For example, the mountinginstructions could be provided via a display device, such as displaydevice 490 of FIG. 4, and may include information such as quantities orpositions of shims (e.g., shims 344, 346, 348, 354, 358 of FIG. 3) orbolts (e.g., bolts 324, 326, 328) to use for mounting the radar units.

At block 504, the method 500 includes determining, for each radar unit,a measured pitch direction and a measured yaw direction based on dataobtained using a photogrammetry system. For example, with reference toFIG. 4, photogrammetry system 480 could measure the yaw and pitchdirections based on captured images of the radar units while the radarunits are mounted to the vehicle. Referring back to FIG. 1 by way ofexample, the photogrammetry system could measure the yaw direction ofthe radar unit (not shown) mounted to location 104 as the yaw direction124, and the yaw direction of the radar unit mounted to location 106 asyaw direction 126, etc.

Thus, for example, based on the data obtained at block 504, a computingdevice may be able to calculate various parameters for each of the radarunits. For example, the computing device may be configured to use thephotogrammetry measurements to determine the orientation of eachmounting structure with respect to each other radar unit or with respectto a reference axis of the vehicle (e.g., axis 112 a, 112 b, etc., ofFIG. 1). As part of the calculations, the computing device may be ableto calculate a measured elevational or pitch direction, and a measuredazimuthal or yaw direction for each of the radar units. The pitchdirection of a radar unit may relate to a rotation of the radar unitwith respect to the horizon (i.e. with respect to the plane in which thevehicle travels). The pitch direction may be described by pitchdirection 372 a of radar unit 312 of FIG. 3, or by yaw direction 362 aof the mounting structure 302 of FIG. 3. The yaw direction of the radarunit may relate to a rotation of the radar unit within the plane of thehorizon (i.e. within the plane in which the vehicle travels). The yawdirection may be described by yaw direction 372 c of the radar unit 312shown in FIG. 3, or by yaw direction 362 c of the mounting structure 302of FIG. 3.

Accordingly, in some embodiments, determining the measured pitch and yawdirections at block 504 could be performed before the mounting of theradar units at block 502. For instance, the photogrammetry system coulddetermine the orientations of mounting structures where the radar unitswill be mounted, and then then the radar units could be mountedappropriately in line with the discussion of FIG. 3 (e.g., with certainquantities of shims, etc.).

At block 506, the method 500 includes determining yaw angles between atleast two of the radar units. A particular yaw angle between twoparticular radar units may be determined based on a difference betweenthe measured yaw directions of the two particular radar units. Referringback to FIG. 1 by way of example, the yaw angle between the radar unitsmounted to locations 104 and 106 could be computed as the differencebetween yaw direction 124 and yaw direction 126, and the yaw anglebetween the radar units mounted to locations 106 and 110, could bedetermined as the difference between yaw direction 126 and yaw direction130, etc.

At block 508, the method 500 includes determining, for each radar unit,a pitch offset and a yaw offset. Once the measured pitch direction andthe measured yaw direction for each of the radar units has beencalculated, the computing device may be able to determine a pitch offsetand a yaw offset for each of the radar units (or mounting structures,etc.). The pitch offset may be the difference between a predeterminedpitch direction the measured pitch direction for example. Thepredetermined pitch direction may be stored in the computing device ormay be received as an input by the computing device, for example.

Further, the yaw offset may be determined based on the differencebetween a predetermined yaw angle and one or more of the measured yawangles. Referring back to FIG. 1 by way of example, the yaw offsetscould be calculated for each radar unit such that the yaw angle betweenyaw directions 124 and 126 is approximately equal to the yaw anglesbetween yaw directions 126, 130 and between yaw directions 130, 128 andbetween yaw directions 128, 124. Alternatively, for example, the yawoffsets could be calculated for each radar unit to achieve certaindesired yaw angles between each pair of radar units. Thus, thepredetermined yaw angle may be based on the number, orientation, andconfiguration of the radar units in line with the discussion above.

In some examples, the offset may be measured in degrees. Additionally,in some further examples, one (or more) of the offsets may be determinedto be zero degrees in examples where one of the radar units (orassociated mounting structures) has a correct alignment with respect toone of the pitch or yaw directions.

In some embodiments, the method 500 may also include determining a sumof the determined yaw angles. In these embodiments, determining the yawoffsets at block 508 may be based also on a difference between a targetsum and the determined sum. Referring back to FIG. 1 by way of example,the sum of the yaw angles from arrows 124 to 126, 126 to 130, 130 to128, and 128 to 124 may be expected to be 360 degrees (i.e., the targetsum). However, in some examples, due to measurement errors from thephotogrammetry system or due to other manufacturing/assemblyvariabilities (e.g., locations 104 and 106 may not be symmetrical,etc.), the actual determined sum may be different from the target (360degree) sum. Thus, in these examples, the method 500 may distribute thedifference between the sums among the yaw offsets determined for eachradar unit. And thus, continuing with the example of FIG. 1, the variousyaw angles after applying the redistributed offsets may more likely beapproximately the same. Further, in some examples, the method 500 mayalso include providing an alert if the difference between the target sumand the determined sum exceeds a threshold. Referring back to FIG. 4 byway of example, the alert could be provided using any of the outputinterfaces 450, or may be included in mounting instructions (e.g.,quantities of shims, etc.) that are provided, for instance, using thedisplay device 490.

At block 510, the method 500 includes for at least one mountingstructure adjusting the radar unit based on at least one of thedetermined pitch offsets and at least one of the determined yaw offsets.In some embodiments, each determined offset may be compared to an offsetthreshold. In some examples, the offset threshold may be the same foreach of the pitch and yaw offsets. However, in other examples, each ofthe pitch and yaw offsets may have a different threshold offset. Inthese embodiments, if the determined offset for the pitch and yawdirections is greater than the threshold, the radar unit and may beadjusted such that each of the pitch and yaw offsets are within therespective threshold.

In some examples, a computing device may provide offset adjustmentinformation (e.g., mounting instructions). The information may beprovided as a number of degrees and the direction in which theadjustment needs to be made. In another example, the computing devicemay report quantities of shims that the respective radar unit needs touse with each bolt, in line with the discussion of FIG. 3. Each quantityof shims determined for a respective bolt could be based on a predefinedshim thickness as well as the yaw offsets and pitch offsets determinedat block 508. The predefined shim thickness could be stored, forexample, in data storage 440 of the device 400 or could be input by auser of the device 400. Further, in some examples, the adjustment atblock 510 may include providing, with the mounting instructionsdescribed at block 502, the determined quantities of shims and/or typesof shims that should be used (e.g., having desired thickness values.

In some embodiments, the method 500 may also include providing an alertif one or more of the determined quantities of shims exceeds athreshold. Referring back to FIG. 3 by way of example, if the number ofshims used for any of the bolts 324-326 exceeds the threshold, then thebolt may bind with the shims, or may not reach the mounting structure302 through the radar unit 312 and the shims due to an extent (e.g.,length) of the bolt. Thus, the adjustment at block 510 could be basedalso on each of the determined quantities of shims being less than thethreshold.

In some embodiments, the method 500 may also include determining anexpected angle between a bolt and a mounting structure based on thedetermined quantities of shims, and providing an alert if the expectedangle exceeds a threshold. Referring back to FIG. 3 by way of example,if the quantity of shims used for the bolt 326 (e.g., one shim 346) isdifferent from the quantities used for any of the other bolts 324, 328,then the expected angle (e.g., angle between dotted line 336 andmounting structure 302) may be affected. And if that expected angleexceeds a threshold (e.g., 3 degrees), then the bolt 326 may bind withthe shim 346, the radar unit 312, and/or the mounting structure 302. Forexample, the bolt 326 may be at an unsuitable angle to enter into thehole 306. Thus, in these embodiments, the method 500 may allow forproviding the alert and/or determining a different combination of shimsthat could achieve the desired offsets while remaining within thethreshold angle between each bolt and the mounting structure.

Further, in some examples, the photogrammetry system may be detectingthe orientations of the radar units/mounting structures while the radarunits are being adjusted. In this example, the photogrammetry system ora computer coupled to the photogrammetry system may provide an alertwhen the mounting structure was adjusted to bring each of the pitch andyaw offsets within the respective offset thresholds.

Further, in some examples, block 510 may include mounting the radar unitto mounting structures in line with the discussion in FIG. 3. The radarunits may be mounted to the mounting structures before or after theradar units are adjusted based on the calculated offsets. Thus, in oneexample, after determining all the pitch offsets and the yaw offsets atblock 508, the radar units may be mounted to the vehicle as described atblock 502, and pointed in a direction within the threshold range of thedesired direction (i.e. orientation) of the radar unit. In anotherexample, where the radar unit is already mounted before block 510, theradar unit could be re-mounted according to adjustment of block 510(e.g., with different quantities of shims, etc.).

It should be understood that arrangements described herein are forpurposes of example only. As such, those skilled in the art willappreciate that other arrangements and other elements (e.g. machines,apparatuses, interfaces, functions, orders, and groupings of functions,etc.) can be used instead, and some elements may be omitted altogetheraccording to the desired results. Further, many of the elements that aredescribed are functional entities that may be implemented as discrete ordistributed components or in conjunction with other components, in anysuitable combination and location.

While various aspects and embodiments have been disclosed herein, otheraspects and embodiments will be apparent to those skilled in the art.The various aspects and embodiments disclosed herein are for purposes ofillustration and are not intended to be limiting, with the scope beingindicated by the following claims.

What is claimed is:
 1. A method comprising: mounting a plurality ofradar units to a vehicle; for each radar unit, determining a measuredpitch direction and a measured yaw direction based on data obtainedusing a photogrammetry system; determining yaw angles between at leasttwo of the radar units, wherein a particular yaw angle between twoparticular radar units is based on a difference between the measured yawdirections of the two particular radar units; for each radar unit:determining a pitch offset based on a difference between a target pitchdirection and the measured pitch direction of the radar unit;determining a yaw offset based at least in part on a difference betweena target yaw angle and one of the determined yaw angles; and adjustingorientation of at least one of the radar units based on at least one ofthe determined pitch offsets and at least one of the determined yawoffsets.
 2. The method of claim 1, wherein, for one or more of the radarunits, determining the yaw offset is based also on a difference betweenthe target yaw angle and another one of the determined yaw angles. 3.The method of claim 1, wherein adjusting the orientation is carried outafter determining the pitch offsets and the yaw offsets for all theradar units.
 4. The method of claim 1, further comprising: determining asum of the determined yaw angles, wherein determining, for each radarunit, the yaw offset is based also on a difference between a target sumand the determined sum, wherein the target sum is 360 degrees.
 5. Themethod of claim 4, further comprising: providing an alert if thedifference between the target sum and the determined sum exceeds athreshold.
 6. The method of claim 1, wherein mounting the plurality ofradar units to the vehicle is via a plurality of mounting structures onthe vehicle, wherein each radar unit is coupled to a respective mountingstructure via a plurality of bolts positioned through the radar unit andthe respective mounting structure, wherein each bolt is coupled to oneor more shims positioned between the radar unit and the respectivemounting structure, the method further comprising: for each boltcoupling the at least one adjusted radar unit to a mounting structure,determining a quantity of shims based on a predetermined shim thickness,the at least one of the determined pitch offsets, and the at least oneof the determined yaw offsets, wherein adjusting the orientationcomprises re-mounting the at least one adjusted radar unit based atleast in part on at least one of the determined quantities of shims. 7.The method of claim 6, further comprising: providing an alert if one ormore of the determined quantities of shims exceeds a threshold, whereinthe re-mounting is based also on each of the determined quantities beingless than the threshold.
 8. The method of claim 6, further comprising:for each bolt coupling the at least one adjusted radar unit to themounting structure, determining an expected angle between the bolt andthe mounting structure based on at least two of the determinedquantities of shims; and providing an alert if the expected angleexceeds a threshold, wherein the re-mounting is based also on theexpected angle being less than the threshold.
 9. A device comprising:data storage storing data generated by a photogrammetry system, the dataindicating measured directions of a plurality of radar units mounted toa vehicle; and a computational unit configured to perform operationscomprising: using the data in the data storage to determine, for eachradar unit, a measured pitch direction and a measured yaw direction;determining yaw angles between at least two of the radar units, whereina particular yaw angle between two particular radar units is based on adifference between the measured yaw directions of the two particularradars; for each radar unit, determining a yaw offset based at least inpart on a difference between a target yaw angle and one of thedetermined yaw angles; for each radar unit, determining a pitch offsetbased on a difference between a target pitch direction and the measuredpitch direction of the radar unit; and providing mounting instructionsfor mounting the radar units, wherein the mounting instructions arebased on the determined pitch offsets and the determined yaw offsets.10. The device of claim 9, wherein the vehicle includes a plurality ofmounting structures, wherein the radar units are coupled to the mountingstructures via a plurality of bolts, wherein each bolt is positionedthrough a respective radar unit and a respective mounting structure,wherein the bolt is coupled to one or more shims positioned between therespective radar unit and the respective mounting structure, and whereinthe operations further comprise: for each of the bolts, determining aquantity of shims based on a predetermined shim thickness, at least oneof the determined pitch offsets, and at least one of the determined yawoffsets.
 11. The device of claim 10, wherein the mounting instructionsare indicative of the determined quantities of shims.
 12. The device ofclaim 10, wherein the operations further comprise generating an alert inresponse to one or more of the determined quantities of shims exceedinga threshold, and wherein the mounting instructions are indicative of thealert.
 13. The device of claim 10, wherein the operations furthercomprise: for each of the bolts, determining an expected angle betweenthe bolt and the respective mounting structure based on at least two ofthe determined quantities of shims; and generating an alert in responseto the expected angle exceeding a threshold, wherein the mountinginstructions are indicative of the alert.
 14. The device of claim 9,further comprising an output interface, wherein providing the mountinginstructions is via the output interface.
 15. The device of claim 9,further comprising: a communication interface, wherein the operationsfurther comprise: receiving the data from the photogrammetry apparatusvia the communication interface; and storing the received data into thedata storage.
 16. An article of manufacture including a non-transitorycomputer-readable medium, having stored thereon program instructionsthat, if executed by a computing device, cause the computing device toperform operations comprising: obtaining data generated by aphotogrammetry system, the data pertaining to a plurality of radar unitsmounted to a vehicle; based on the obtained data, determining measuredyaw angles between at least two of the radar units, wherein a particularmeasured yaw angle between two particular radar units is based on adifference between yaw directions measured for the two particular radarunits; for each radar unit: determining a yaw offset based at least inpart on a difference between a target yaw angle and one of the measuredyaw angles; determining a measured pitch direction based on the obtaineddata; and determining a pitch offset based on a difference between atarget pitch direction and the measured pitch direction.
 17. The articleof manufacture of claim 16, wherein obtaining the data comprisesrequesting the data from the photogrammetry system.
 18. The article ofmanufacture of claim 16, wherein obtaining the data comprises retrievingthe data from a data storage of the computing device.
 19. The article ofmanufacture of claim 16, wherein, for one or more of the radar units,determining the yaw offset is based also on a difference between thetarget yaw angle and another one of the determined yaw angles.
 20. Thearticle of manufacture of claim 16, wherein the operations furthercomprise: providing mounting instructions for mounting the radar units,wherein the mounting instructions are based on the determined pitchoffsets and the determined yaw offsets.